19 research outputs found

    Evolutionary support vector machines for time series forecasting

    Get PDF
    Abstract. Time Series Forecasting (TSF) uses past patterns of an event in order to predict its future values and is a key tool to support decision making. In the last decades, Computational Intelligence (CI) techniques, such as Artificial Neural Networks (ANN) and more recently Support Vector Machines (SVM), have been proposed for TSF. The accuracy of the best CI model is affected by both the selection of input time lags and the model’s hyperparameters. In this work, we propose a novel Evolutionary SVM (ESVM) approach for TSF based on the Estimation Distribution Algorithm to search for the best number of inputs and SVM hyperparameters. Several experiments were held, using a set of six time series from distinct real-world domains. Overall, the proposed ESVM is competitive when compared with an Evolutionary ANN (EANN) and the popular ARIMA methodology, while consuming less computational effort when compared with EANN.The research reported here has been supported by FEDER (program COMPETE and FCT) under project FCOMP-01-0124-FEDER-02267

    Evolutionary optimization of sparsely connected and time-lagged neural networks for time series forecasting

    Get PDF
    Time Series Forecasting (TSF) is an important tool to support decision mak- ing (e.g., planning production resources). Artificial Neural Networks (ANN) are innate candidates for TSF due to advantages such as nonlinear learn- ing and noise tolerance. However, the search for the best model is a complex task that highly affects the forecasting performance. In this work, we propose two novel Evolutionary Artificial Neural Networks (EANN) approaches for TSF based on an Estimation Distribution Algorithm (EDA) search engine. The first new approach consist of Sparsely connected Evolutionary ANN (SEANN), which evolves more flexible ANN structures to perform multi-step ahead forecasts. The second one, consists of an automatic Time lag feature selection EANN (TEANN) approach that evolves not only ANN parameters (e.g., input and hidden nodes, training parameters) but also which set of time lags are fed into the forecasting model. Several experiments were held, using a set of six time series, from different real-world domains. Also, two error metrics (i.e., Mean Squared Error and Symmetric Mean Absolute Per- centage Error) were analyzed. The two EANN approaches were compared against a base EANN (with no ANN structure or time lag optimization) and four other methods (Autoregressive Integrated Moving Average method, Random Forest, Echo State Network and Support Vector Machine). Overall, the proposed SEANN and TEANN methods obtained the best forecasting results. Moreover, they favor simpler neural network models, thus requiring less computational effort when compared with the base EANN.The research reported here has been supported by the Spanish Ministry of Science and Innovation under project TRA2010-21371-C03-03 and FCT - Fundacao para a Ciencia e Tecnologia within the Project Scope PEst- OE/EEI/UI0319/2014. The authors want to thank specially Martin Stepnicka and Lenka Vavrickova for all their help. The authors also want to thank Ramon Sagarna for introducing the subject of EDA

    Diseño automático de redes de neuronas artificiales para la predicción de series temporales

    Get PDF
    El ser humano ha avanzado mucho, tecnológicamente hablando, en el último siglo. La sed por descubrir e innovar no tiene límites y cómo no, aplicar dichas innovaciones para nuestro provecho y bienestar general. Uno de los campos de investigación a los que se ha aplicado dicha innovación es la predicción. Al hablar de predicción, lo primero que nos puede venir a la cabeza son temas no tan científicos como la astrología o la lectura de manos, pero en realidad, diversos métodos estadísticos y matemáticos pueden ayudar a proporcionar información sobre el futuro. Uno de los ejemplos más comunes es la predicción del tiempo meteorológico que podemos observar cada día en televisión. Además de los métodos ya mencionados, en los últimos años ha proliferado el estudio de la predicción mediante técnicas de inteligencia computacional y dentro de las predicciones, aquellas que se ocupan de predecir series temporales. La predicción de series temporales consiste en llevar a cabo aproximaciones o estimaciones de qué valores tendrán los elementos futuros de una serie temporal partiendo de los valores de los elementos previos o ya conocidos. Como veremos en esta tesis doctoral, a lo largo de los años se han usado diferentes técnicas de inteligencia computacional con este propósito, aunque nosotros nos centraremos en las redes de neuronas artificiales. Plantearemos las ventajas y problemas que se pueden dar y nos centraremos en intentar solventar dichos problemas. Uno de los problemas clave que se plantea actualmente a la hora de aplicar redes de neuronas artificiales a cualquier dominio dado, es su correcto diseño. Estudiaremos pues las diferentes soluciones propuestas para el correcto diseño de las redes de neuronas artificiales, aunque terminaremos centrándonos en aquellas que hacen uso de la computación evolutiva. Este modelo de redes es el que se conoce como redes de neuronas artificiales evolutivas. Esta tesis doctoral presenta tres enfoques diferentes para el modelado automático de redes de neuronas artificiales. Cada enfoque irá destinado a solventar cada uno de lo que nosotros consideramos los tres puntos o problemas claves existentes al diseñar una red de neuronas artificial. El primer enfoque consistirá en el tratamiento de los datos que son pasados como patrones a la red para que ésta aprenda y sea evaluada. El segundo enfoque se centrará en las diferentes técnicas evolutivas que pueden ser usadas, cómo obtener un fenotipo a partir de un genotipo (y viceversa) y cómo evaluar una red. El último enfoque que se estudiará, es el tipo de arquitectura de red que debe ser usada para la predicción de series temporales. El objetivo final de esta tesis doctoral es llevar a cabo un sistema automático de diseño de redes de neuronas artificiales para solventar problemas de predicción de series temporales con la mayor exactitud posible y transparente al usuario, es decir, que este no tenga que ser un experto en la materia para poder hacer uso de él. --------------------------------------------------------------------------------------------------------------------------------------------------------------------------The human being have progressed a lot, technologically speaking, in the last century. The thirst for discovery and innovation has no limits and of course, to apply these innovations to our benefit and general welfare. One of the research areas that have been applied to this innovation is the prediction. When we talk about prediction, the first thing that may come to our minds are not so scientific issues as astrology or hand reading, but in fact, several statistical and mathematical methods can help to provide information about the future. One of the most common examples is the weather forecasting that we can watch every day on television. Besides the methods already commented, in recent years it has proliferated the prediction study using computational intelligence techniques and within these predictions, those consisting of time series forecasting. Time series forecasting consist of carrying out approximations or estimations about which values will have the future elements of a time series starting from the values of the previous already known elements. As we discuss in this PhD thesis, over the years it has been used different computational intelligence techniques for this purpose, although we will focus on artificial neural networks.We will present the advantages and problems that may appear and we will focus on trying to solve these problems. One of the key problems that currently arise in applying artificial neural networks to any given domain, is its correct design. We will study then the different solutions proposed for the proper design of artificial neural networks, although at the end, we will focus on those which use evolutionary computation. This network model is known as evolutionary artificial neural networks. This PhD thesis presents three different approaches for the automatic design of artificial neural networks. Each approach will be dedicated to solve each of what we consider the three points or key problems in designing an artificial neural network. The first approach will consist of treating the data that are passed to the network as patterns to make it learn and be evaluated. The second approach will focus on the different evolutionary techniques that can be used, how to obtain a phenotype from a genotype (and vice versa) and how to evaluate a network. The last approach to be studied, is the type of network architecture to be used for time series forecasting. The ultimate goal of this thesis is to implement an automatic system to design artificial neural networks to solve time series forecasting problems as accurately as possible and transparent to the user, i.e. that the user did not have to be an expert to make use of it

    Evolving artificial neural networks applied to generate virtual characters

    Get PDF
    Computer game industry is one of the most prof­ itable nowadays. Although this industry has evolved fast in the last years in different fields, Artificial Intelligence (AI) seems to be stuck. Many games still make use of simple state machines to simulate AI. New models can be designed and proposed to replace this jurassic technique. In this paper we propose the use of Artificial Neural Networks (ANN) as a new model. ANN will be then in charge of receiving information from the game (sensors) and carry out actions (actuators) according to the information received. The search for the best ANN is a complex task that strongly affects the task performance while often requiring a high computational time. In this work, we present ADANN, a system for the automatic evolution and adaptation of artificial neural networks based on evolutionary ANN (EANN). This approach use Genetic Algorithm (GA) that evolves fully connected Artificial Neural Network. The testing game is called Unreal Tournament 2004. The new agent obtained has been put to the test jointly with CCBot3, the winner of BotPrize 2010 competition [1], and have showed a significant improvement in the humannesss ratio. Additionally, we have confronted our approach and CCBot3 (winner of BotPrize competition in 2010) to First-person believability assessment (BotPrize original judging protocol), demonstrating that the active involvement of the judge has a great impact in the recognition of human-like behaviour

    Forecasting seasonal time series with computational intelligence: on recent methods and the potential of their combinations

    Get PDF
    Accurate time series forecasting is a key issue to support individual and or- ganizational decision making. In this paper, we introduce novel methods for multi-step seasonal time series forecasting. All the presented methods stem from computational intelligence techniques: evolutionary artificial neu- ral networks, support vector machines and genuine linguistic fuzzy rules. Performance of the suggested methods is experimentally justified on sea- sonal time series from distinct domains on three forecasting horizons. The most important contribution is the introduction of a new hybrid combination using linguistic fuzzy rules and the other computational intelligence methods. This hybrid combination presents competitive forecasts, when compared with the popular ARIMA method. Moreover, such hybrid model is more easy to interpret by decision-makers when modeling trended series.The research was supported by the European Regional Development Fund in the IT4Innovations Centre of Excellence project (CZ.1.05/1.1.00/02.0070). Furthermore, we gratefully acknowledge partial support of the project KON- TAKT II - LH12229 of MSˇMT CˇR

    Time series forecasting using a weighted cross-validation evolutionary artificial neural network ensemble

    Get PDF
    The ability to forecast the future based on past data is a key tool to support individual and organizational decision making. In particular, the goal of Time Series Forecasting (TSF) is to predict the behavior of complex systems by looking only at past patterns of the same phenomenon. In recent years, several works in the literature have adopted Evolutionary Artificial Neural Networks (EANNs) for TSF. In this work, we propose a novel EANN approach, where a weighted n-fold validation fitness scheme is used to build an ensemble of neural networks, under four different combination methods: mean, median, softmax and rank-based. Several experiments were held, using six real-world time series with different characteristics and from distinct domains. Overall, the proposed approach achieved competitive results when compared with a non-weighted n-fold EANN ensemble, the simpler 0-fold EANN and also the popular Holt–Winters statistical method.This work was supported by University Carlos III of Madrid and by Community of Madrid under project CCG10-UC3M/TIC-5174. The work of P. Cortez was funded by FEDER (program COMPETE and FCT) under project FCOMP-01-0124-FEDER-022674

    Evolving sparsely connected neural networks for multi-step ahead forecasting

    Get PDF
    Time Series Forecasting (TSF) is an important tool to sup- port decision making. Artificial Neural Networks (ANN) are innate candidates for TSF due to advantages such as nonlin- ear learning and noise tolerance. However, the search for the best ANN is a complex task that highly affects the forecast- ing performance. In this paper, we propose a novel Sparsely connected Evolutionary ANN (SEANN), which evolves more flexible ANN structures to perform multi-step ahead fore- casts. This approach is compared with a similar strategy but that only evolves fully connected ANNs (FEANN) and a conventional TSF method (i.e. ARIMA methodology). A set of six time series, from different real-world domains, was used in the comparison. Overall, the obtained results re- veal the proposed SEANN approach as the best forecasting method, optimizing more simpler structures and requiring less computational effort when compared with the fully con- nected evolutionary ANN strategy.Community of Madrid under project CCG10-UC3M/TIC-5174

    Forecasting seasonal time series with computational intelligence: contribution of a combination of distinct methods

    Get PDF
    Accurate time series forecasting are important for displaying the manner in which the past contin- ues to affect the future and for planning our day to day activities. In recent years, a large litera- ture has evolved on the use of computational in- telligence in many forecasting applications. In this paper, several computational intelligence techniques (genetic algorithms, neural networks, support vec- tor machine, fuzzy rules) are combined in a distinct way to forecast a set of referenced time series. Fore- casting performance is compared to the a standard and method frequently used in practice.Project DAR 1M0572 of the MÅ MT ÄŒR

    Evolving time-lagged feedforward neural networks for time series forecasting

    Get PDF
    Time Series Forecasting (TSF) is an important tool to sup- port both individual and organizational decisions. In this work, we propose a novel automatic Evolutionary Time- Lagged Feedforward Network (ETLFN) approach for TSF, based on an Estimation Distribution Algorithm (EDA) that evolves not only Artificial Neural Network (ANN) parame- ters but also which set of time lags are fed into the fore- casting model. Such approach is compared with similar strategy that only selects ANN parameter and the conven- tional TSF ARIMA methodology. Several experiments were held by considering six time series from distinct domains. The obtained multi-step ahead forecasts were evaluated us- ing SMAPE error criteria. Overall, the proposed ETLFN method obtained the best forecasting results. Moreover, it favors simpler neural network models, thus requiring less computational effort.University Carlos IIICommunity of Madrid under project CCG10- UC3M/TIC-5174

    Artificial intelligence approaches for the generation and assessment of believable human-like behaviour in virtual characters

    Get PDF
    Having artificial agents to autonomously produce human-like behaviour is one of the most ambitious original goals of Artificial Intelligence (AI) and remains an open problem nowadays. The imitation game originally proposed by Turing constitute a very effective method to prove the indistinguishability of an artificial agent. The behaviour of an agent is said to be indistinguishable from that of a human when observers (the so-called judges in the Turing test) can not tell apart humans and non-human agents. Different environments, testing protocols, scopes and problem domains can be established to develop limited versions or variants of the original Turing test. In this paper we use a specific version of the Turing test, based on the international BotPrize competition, built in a First-Person Shooter video game, where both human players and non-player characters interact in complex virtual environments. Based on our past experience both in the BotPrize competition and other robotics and computer game AI applications we have developed three new more advanced controllers for believable agents: two based on a combination of the CERA-CRANIUM and SOAR cognitive architectures and other based on ADANN, a system for the automatic evolution and adaptation of artificial neural networks. These two new agents have been put to the test jointly with CCBot3, the winner of BotPrize 2010 competition [1], and have showed a significant improvement in the humanness ratio. Additionally, we have confronted all these bots to both First-person believability assessment (BotPrize original judging protocol) and Third-person believability assess- ment, demonstrating that the active involvement of the judge has a great impact in the recognition of human-like behaviour.MICINN -Ministerio de Ciencia e Innovación(FCT-13-7848
    corecore